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Walter  H.  Ku,  Member,  IEEE 


Abstract — This  paper  studies  the  comparative  tracking  per¬ 
formance  of  the  recursive  least  squares  (RLS)  and  least  mean 
square  (LMS)  algorithms  for  time-varying  inputs,  specifically 
for  linearly  chirped  narrowband  input  signals  in  additive  white 
Gaussian  noise.  It  is  shown  that  the  structural  differences  in  the 
implementation  of  the  LMS  and  RLS  weight  updates  produce 
regions  where  the  LMS  performance  exceeds  that  of  the  RLS  and 
other  regions  where  the  converse  occurs.  These  regions  are  shown 
to  be  a  function  of  the  signal  bandwidth  and  signal-to-noise  ratio 
(SNR).  LMS  is  shown  to  place  a  notch  in  the  signal  band  of  the 
mean  lag  filter,  thus  reducing  the  lag  error  and  improving  the 
tracking  performance.  For  the  chirped  signal,  it  is  shown  that  this 
produces  smaller  tracking  error  for  small  SNR.  For  high  SNR, 
there  is  a  region  of  signal  bandwidth  for  which  RLS  will  provide 
lower  error  than  LMS,  but  even  for  these  high  SNR  inputs,  LMS 
always  provides  superior  performance  for  very  narrowband 
signals. 

Index  Terms — Adaptive  filters,  autoregressive  model,  least  mean 
square,  recursive  least  squares,  tracking. 

1.  Introduction 

THE  tracking  behavior  of  adaptive  filtering  algorithms  is  a 
fundamental  issue  in  defining  their  performance  in  nonsta¬ 
tionary  operating  environments.  It  has  been  established  [l]-[3] 
that  adaptive  algorithms  that  exhibit  good  convergence  proper¬ 
ties  in  stationary  environments  do  not  necessarily  provide  good 
tracking  performance  in  a  nonstationary  environment  because 
the  convergence  behavior  of  an  adaptive  filter  is  a  transient  phe¬ 
nomenon,  whereas  the  tracking  behavior  is  a  steady-state  prop¬ 
erty. 

There  are  two  fundamentally  different  nonstationary  sce¬ 
narios  that  have  been  previously  analyzed.  The  first  involves 
a  time-varying  system  where  the  cross-correlation  vector 
between  the  input  signal  to  the  adaptive  filter  and  the  desired 
response  is  time  varying;  the  second  involves  a  nonstationary 
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input  to  the  adaptive  filter.  The  first  case  occurs  in  system 
identification  applications  and  is  analyzed  in  [1],  [2],  and  [4]. 
The  latter  case  occurs  in  adaptive  equalization,  signal  recovery, 
and  other  applications  and  has  been  analyzed  in  [3],  [5],  and 
[6]  for  the  least  mean  square  (LMS)  and  the  recursive  least 
squares  (RLS)  adaptive  algorithms  for  the  spceifie  case  of  a 
deterministic  chirped  signal  input  to  an  adaptive  prediction 
filter.  This  signal  model  provides  a  constant  nonstationarity  that 
is  useful  in  separating  the  convergence  and  tracking  behavior. 
The  results  show  that  LMS  usually  provides  better  tracking 
behavior  than  exponentially  weighted  RLS  for  linearly  chirped 
sinusoids  in  additive  white  Gaussian  noise  (AWGN).  This 
behavior  is  because  LMS  is  model  independent,  whereas  RLS 
must  employ  a  model  of  the  data  correlation  matrix  (such  as  an 
exponential  weighting),  which  may  not  match  the  characteris¬ 
tics  of  the  input  signals.  It  is  further  shown  in  [7]  that  superior 
tracking  performance  is  obtained  by  RLS  if  the  extended  RLS 
(ERLS)  algorithm  with  a  time-varying  state  transition  matrix  is 
used.  For  chirped  inputs,  the  ERLS  implementation  estimates 
the  chirp  rate  from  the  data  and  uses  it  thereafter  in  the  adaptive 
update  equations  [7].  Nevertheless  the  exponentially  weighted 
RLS  algorithm  is  frequently  employed  in  practical  systems 
due  to  its  computational  simplicity  compared  with  ERLS  and 
because,  in  many  scenarios,  an  accurate  state-space  model 
is  not  available.  Consequently,  a  comparison  of  the  tracking 
performance  of  the  LMS  and  the  exponentially  weighted  RLS 
algorithm  remains  an  important  issue 

The  tracking  behavior  of  the  RLS  adaptive  filter  was  extended 
to  include  stochastic  narrowband  input  signals  in  [8].  The  signal 
bandwidth  is  shown  to  be  an  important  parameter  in  optimizing 
the  RLS  filter  performance  [8].  This  paper  extends  the  analysis 
in  [3],  [5],  [6],  and  [8]  to  define  the  tracking  performance  of  the 
LMS  adaptive  algorithm  for  narrowband  stochastic  input  signals 
using  the  same  signal  model  used  in  [8].  This  will  allow  the 
comparative  performance  of  the  LMS  and  RLS  algorithms  to  be 
defined  for  a  broader  class  of  input  signals  and  to  determine  the 
effects  of  signal  bandwidth  on  the  tracking  performance.  This  is 
shown  to  provide  a  number  of  interesting  results  for  the  adaptive 
signal  recovery  application. 

The  structural  differences  in  the  LMS  and  RLS  weights  up¬ 
dates  arc  shown  to  produce  regions  where  the  LMS  performance 
exceeds  that  of  the  RLS  and  vice  versa.  These  regions  arc  func¬ 
tions  of  the  signal  bandwidth  and  signal-to-noise  ratio  (SNR). 
LMS  is  shown  to  place  a  notch  in  the  signal  band  of  the  mean 
lag  filter,  thus  reducing  the  lag  error  and  improving  the  tracking 
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performance.  The  optimal  adaptation  constant  for  both  LMS  and 
RLS  with  chirped  narrowband  inputs  is  shown  to  yield  a  lag 
misadjustment  error  that  is  half  the  noise  misadjustment  error, 
which  is  identical  to  previous  results  for  deterministic  inputs 
[3],  [5].  This  is  in  contrast  to  the  result  obtained  for  systems 
with  time-varying  coefficients  [4],  where  optimal  tracking  oc¬ 
curs  when  the  lag  and  noise  misadjustment  are  equal 

II.  LMS  Algorithm 

In  order  to  facilitate  the  steady-state  tracking  performance 
comparison  between  the  LMS  and  RLS  algorithms  for  a  chirped 
narrowband  signal  in  AWGN,  the  problem  introduction,  the 
ehirped  narrowband  signal  model,  the  optimum  predictor,  and 
notations  used  in  this  paper  will  be  the  same  as  described  in 
[8],  where  the  steady-state  tracking  performance  of  the  RLS 
algorithm  is  studied  and  contrasted  with  the  results  in  [3].  To 
complete  the  comparisons,  we  adopt  the  notational  conventions 
in  the  previous  studies  [3],  [5],  [6],  and  [8] 

The  LMS  adaptive  one-step  forward  predictor  is  summarized 
by  the  following  filter  output  and  weight  update  equations.  The 
filter  output  is  given  by 

Vk  =  Wl_lxk  (1) 

and  the  weight  update  equation  is 

H4.  =  Wfc-I  +  t-ix] t  (dk  -  yk)  ■  (2) 

The  LMS  algorithm  uses  the  instantaneous  estimate  of  the  input 
correlation  and  cross  correlation  (x£c4)  to  update  the 

filter  weights  [2]. 

The  LMS  weight  update  equation  is  used  to  decompose  the 
weight  error  into  a  mean  and  a  fluctuation  component.  The  dif¬ 
ference  between  the  LMS  weights  and  the  optimum  Wiener 
weights  is  [5] 

Afc_i  =  W4-i  -  W£.  (3) 

The  adaptive  filter  output  misadjustment  is  defined  as 

Mk  =  E  |ALi«fc|2.  (4) 

The  LMS  filter  weight  update  equation  can  be  rewritten  as 

Wfc  =  [I  -  fixlxl]Wk-i  +  AtXfcXfc  (5) 

where  the  desired  signal  dk  in  this  ease  is  equal  to  x Sub¬ 
tracting  from  both  sides  yields 

A*  =  (I  -  ,ixlxl) Afc_x  -  Th  +  ixxlel  (6) 

where  T is  the  ehange  in  the  optimum  weight  vector  at  succes¬ 
sive  iteration 

Tfc  =  W£+l  -  W£  =  Vk+iA{W°  ©  D)  (7) 

A  is  a  diagonal  matrix  given  by 


and  V  is  the  chirp  matrix,  D  is  the  signal  direction  veetor  as 
defined  in  [8] 

V  =diag(^,^2,.  (9) 

D  =  [S2*-1/2,  ii2*-22/^ jjMy-MVa]  r  (10) 

where  ^  =  e^,7/;  is  the  ehirp  rate,  and  il  =  defines 

the  initial  eenter  frequency  of  the  speetrum.  The  predietor  error 
when  using  the  optimum  weights  is 

e°k=xk-{W°k)Txk.  (11) 

Employing  the  same  assumptions  utilized  in  [5]  that  x k  is  a 
sequence  of  independent  veetors,  it  follows  that  the  optimum 
prediction  error  e°k  is  uncorrelated  with  the  input,  i.e., 

£{[**-TOr^]4}=o  (i2) 

and  thus,  the  noise  and  lag  weight  errors  are  independent.  Sinee 

the  weight  error  update  equation  is  linear,  it  ean  be  decomposed 
into  the  constituent  terms 

A*-  =  A*  +  A*  (13) 

consisting  of  a  noise  weight  error  veetor  and  a  lag  weight  error 
veetor  with  the  following  update  equations: 

=  [I  -  ftxt xl  ]  AjJ  :  4-  ftxlek  (14) 

Ai  =  [I  -  pxixl]  AUl  -  Tk.  (15) 

Similar  to  the  analysis  for  the  R  LS  algorithm  [8],  the  lag  weight 
error  ean  be  further  decomposed  into  a  mean  component  £[A{J 
and  a  fluctuation  component  A^.  =  A^  —  £[AjJ.  It  will  be 
shown  that  under  the  “slow  adaptation”  condition  employed 
in  [3]  and  [5],  the  misadjustment  from  the  lag  fluctuation  Alk 
is  small  compared  with  the  mean  lag  component.  Thus,  the 
filter  ean  be  viewed  as  three  independent  filters,  consisting  of  a 
Wiener  filter,  noise  filter,  and  lag  filter  component.  The  output 
misadjustment  therefore  consists  of  the  noise,  mean  lag,  and 
lag  fluctuation  misadjustment 

A^,.  =  £[(Afc_1),/^A,._1]  =M'£+(Ml  +  M‘k)  (16) 

where  <££  is  the  input  autocorrelation  matrix,  whieh  ean  be 
written  as  [8] 

*J=£[*;®f]=PnVfc2>Vfc  (17) 

and 

V=[l  +  pRQ(DD’1)]  (18) 

where  p  =  Ps/Pn  is  the  input  SNR.  To  make  further  compar¬ 
isons,  we  will  now  define  a  normalized  adaptation  constant  for 
the  LMS  to  remove  the  units  of  power 

v  -  Pn(  1  +  p)p 


A  =  I  —  V 


(8) 


(19) 
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In  the  following  derivation  of  the  misadjustment,  we  will  as¬ 
sume  the  same  “slow  ehirping”  and  “slow  adaptation”  assump¬ 
tion  as  used  in  [3]  and  [5],  i.e., 

SC  :  Mip  (20) 

SA:A'/i/<2.  (21) 


A.  Noise  Misadjustment 

Sinee  the  noise  and  lag  components  are  uneorrelated,  they  can 
be  evaluated  separately.  The  noise  weight  error  update  in  ( 1 4)  is 
the  same  as  the  stationary  results  [2].  The  steady-state  (k  — *  oo) 
noise  misadjustment  is 


Mn 


Mu 

2  -  Mu° 


(22) 


when 

0  <  /t  <  y—  (23) 

to  assure  the  eonvergenee  in  the  mean  squared  sense,  where 
Amax  is  the  largest  of  all  the  eigenvalues  of  the  input  correla¬ 
tion  matrix  A»,  i  =  1,2, ... ,  M .  By  noting  that  YliLi  A;  = 
Tr(<££)  =  MPn{  1  T  p),  it  ean  be  seen  that  this  condition  is 
easily  satisfied  through  the  slow  adaptation  assumption,  i.e., 
pTr($£)  =  pMPn{l  +  p)  =  Mu  <  2. 


B.  Mean  Lag  Misadjustment 

The  lag  misadjustment  is  eomposed  of  two  parts:  the  mean 
lag  and  the  lag  fluctuation.  The  mean  lag  weights  £[A£]  are 
given  by  the  recursion 

E[Alk)  =\l-^l}E[A‘k_l]-Tk 

=v*  (i-ziA^v-^ALi] 

-  Vfc( V  -  I)  [W°  0  D) .  (24) 

Letting  Qk  =  V-^fc+l^£[AJ.],  the  dependence  of  £[Ay  on 
time  k  can  be  removed,  and  the  above  simplifies  to 

Qk  =  V*  (I  -  txPnV)  Qk- 1  -  (I  -  V*)  (W°  0  D)  .  (25) 


This  is  a  first-order  difference  equation  with  constant  coeffi¬ 
cients.  When  (I  —  pPnV)~l  exists,  Qk  converges  to  a  steady- 
state  value  given  by 

Q  =  -  (A  +  nPnV)-1  a  (W°  ©  D) .  (26) 

Using  the  solution  for  Qk  and  solving  for  f^AjJ  gives  the  mean 
lag  weight  vector  in  the  steady  state  as 

E  [Ay  =  -  V*+1  (A  +  iiP„V)~l  A  (W°  ©  D) 

^--Lv^r'A  {W°QD).  (SC)  (27) 

t-lp n 

The  lag  misadjustment  error  is  given  by 

M[  =  Tr  (E  [A'._i]  "  *l.E  [ALi])  =  PnQ"-J>Qk-i. 

(28) 


At  steady  state  ( k  — ♦  oo),  for  ip  fiPn(  1  -F  p) 

Ml  «Fn  (W°  ©  D)"  A*  (txPnV)-1  VQiPnV)~l 
•A  (W°  Q  D)  =^-rf>2  (29) 


where 


«LMS  =  \Pn{  1  +  p)]2 


(  avu°)  "  ($*)_1  (Ar) 

Ip2 


(30) 


is  the  normalized  lag  misadjustment  of  LMS  for  a  ehirped 
proeess.  In  (30),  is  the  autoeorrelation  matrix  of  the  corre¬ 
sponding  stationary  input  proeess  [8].  Note  that  klms  has  the 
units  of  power  and  is  independent  of  the  chirp  rate  under  the 
slow  chirp  eondition  sinee  the  diagonal  matrix  A  is  linear  with 
respect  to  ip  with  elements  An  «  jipl ,  /  €  [1,2,...  M]. 


C.  Lag  Fluctuation  Misadjustment 

In  this  section,  it  is  shown  that  the  misadjustment  caused  by 
the  lag  fluctuation  is  small  eompared  with  the  mean  lag  misad¬ 
justment.  The  lag  fluctuation  misadjustment  is  given  by 

A4'.=Tr(^Z*)  (31) 


where 


z  k  =  E 


(32) 


is  the  lag  fluctuation  weight  correlation  matrix,  and  the  lag  fluc¬ 
tuation  weight  is  given  by 

A'  =A'  -  E  [A'] 

=  (I  -  l^'kxl)  A[._i  +  /*  -  x'kzl)  E  [ALi]  • 

(33) 


For  the  linear  chirp  model  defined  above,  wc  ean  assume  that  Z* 
takes  the  form  Zk  =  VkBkV*k .  This  allows  the  lag  fluctuation 
misadjustment  to  be  simplified  to 

Mlk=Pnlt{VBk).  (34) 

Using  (33)  and  the  identity  in  the  Appendix,  the  lag  fluctuation 
weight  correlation  Zk  follows  the  update  equation 

z,+  i  —Zk  —  p  (&kZk  -f  Zfc3>£) 

+  M3{^Zfc^  +  ^Tr[^Zfc]} 

+  ,i2E  [A'fc_i] "  *IE  [ALi]  n  (35) 

Substituting  the  form  Zk  =  NkBk\r*k ,  then  (35)  beeomes 

VBfc+iV*  =  Bk  -  fiPn  (BkV  A-  VBk) 

WPn  {DBkV  +  PTr  p VBk ]}  +  fi2PnMlkV.  (36) 

At  steady  state  (k  — >  oo),  Bk  — >  B.  A  bound  of  Tr(Z>B)  can 
be  obtained  by  taking  the  traee  and  then  applying  the  two-norm 
properties  of  the  eorrelation  matriecs  to  give 

(2 tiPn  -  /i2P2Tr[F]}  Tr[VB)  =fx2  P2Tt[DBV] 

+  fi2PnTr[D]Mlk.  (37) 
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TABLE  I 

Summary  of  Optimum  Normalized  Adaptation  Constant  and  Misadjustment 


Parameter 

RLS  Algorithm 

LMS  Algorithm 

Normalized  adaptation 
constant  ( f3 ,  v ) 

II 

v  =  P„(l  +  p)B 

*”rls  »  *LMS 

(AW0)w<D'(AW0)V 

[P„(l  +  p)]Wj^<X»T'(AW0)/^ 

Misadjustment 

M 

2  P  ?  ™ 

Optimal  normalized 
adaptation  constant 

B  JV  SuT 
(m  l  J 

v  =(v*-LMSr 

*  l  M  J 

Optimal  misadjustment 
Alin 

and 


The  optimum  misadjustment  is 


{2  -  nI\Tr[D]}  Tr[PB]  =liPnlr[VBV\  +  ftMlkTr[D\ 
<,iPn  ||P||Tr[BP] 

+  HTr[D]Mlk.  (38) 


Thus 


Ml  =  M‘k_^  =  PnTr[PB]  < 


nPnlt[V\Ml 


< 


2  -  nPn( Tr[P]  +  ||P||) 

fiPnTr[V]Ml 


-2  -  2/iPnTr[P] ' 


The  ratio  of  steady-state  lag  fluctuation  misadjustment  versus 
mean  lag  fluctuation  is  thus 


3 

•Mm  in  —  ~^N1^0U opt*  (42) 

This  has  the  same  form  as  the  RLS  result  derived  in  [8].  In  both 
cases,  the  lag  misadjustment  is  equal  to  half  the  noise  misad¬ 
justment.  The  main  results  obtained  for  the  optimal  normalized 
adaptation  constants  and  misadjustment  noise  for  the  LMS  al¬ 
gorithm  obtained  above  and  the  RLS  algorithm  in  [8]  are  sum¬ 
marized  in  Table  I.  It  is  evident  from  Table  I  that  the  main  differ¬ 
ence  in  performance  between  the  LMS  and  RLS  algorithms  will 
be  defined  by  the  differences  in  kLMs  and  krls.  These  terms 
define  the  properties  of  the  weight  update  structure  of  each  algo¬ 
rithm.  The  performance  differences  that  result  will  be  evaluated 
later  for  the  chirped  AR1  signal  model. 


Ml  ^  l>PnJr[V\ 

M‘  ~  2  -  2/tPnTr[P] ' 


(39) 


By  the  “slow  adaptation”  assumption  (21),  i.e.,  pPnTr[D\  2, 
the  above  ratio  is  much  smaller  than  unity.  Thus,  the  lag  fluc¬ 
tuation  can  be  neglected.  This  is  similar  in  form  to  the  solution 
for  the  chirped  sinusoid  problem  [5]. 

D.  Optimum  Normalized  Adaptation  Constant 

Using  the  previous  results  for  the  noise  and  lag  misadjust¬ 
ment,  the  normalized  adaptation  constant  v  defined  in  (19)  can 
be  optimized  to  minimize  the  filter  misadjustment.  The  total 
misadjustment  is  given  by 


III  Comparative  Tracking  Performance  of  the  LMS 
and  RLS  Algorithms  for  a  Chirped  ARl  Signal  in  Noise 

To  illustrate  the  performance  differences  between  the  LMS 
and  RLS  algorithms  for  a  chirped  nonzero  bandwidth  signal  in 
AWGN,  the  results  obtained  previously  are  applied  to  a  chirped 
ARl  process  embedded  in  AWGN  [8].  The  ARl  process  ean  be 
used  to  model  many  narrowband  signals  sueh  as  binary  phase- 
shift  keying  (BPSK),  as  discussed  in  [8]  and  [9], 

As  shown  in  [8]  for  RLS  and  (6)  for  LMS,  in  the  ALE  config¬ 
uration,  the  weight  error  vector  of  the  LMS  and  RLS  algorithms 
is  given  by 

.  r  (I  -  nxlxl)  A,_x  +  ,ixlel  -  Tk,  LMS 

\  AR-1!^  A*_!  +  Rklrke°k  -  Tk,  RLS  1  ’ 


M  = 


Mu  i>2 

- - ri~^o  H — o  klms* 

2  —  Mu  u2 


(40) 


When  u  1,  the  misadjustment  has  the  same  form  as  the  RLS 
algorithm  defined  in  [8].  Solving  for  the  minimum  with  respect 
to  u ,  the  optimum  normalized  adaptation  constant  is 


!yopt 


(  4t/>2KLMsV/3 

V  ) 


(41) 


where  R*  is  the  RLS  estimate  of  the  input  correlation  matrix 
using  an  exponentially  weighted  estimate  with  weighting  pa¬ 
rameter  A  =  1  —  ft.  The  first  term  summarizes  the  form  of  the 
update  of  the  algorithm.  The  second  term  gives  the  adaptation 
noise  error.  The  last  term  gives  the  lag  error.  The  major  differ¬ 
ence  is  that  the  RLS  algorithm  deeorrelates  the  input  using  R^7 1 . 
For  the  LMS  algorithm,  however,  the  weight  error  update  uses 
successive  updates  to  subtract  components  of  the  signal  from  the 
weight  error.  In  the  RLS  algorithm,  assuming  that  ^  Rl-i, 
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the  previous  weight  is  merely  scaled.  The  weight  error  update 
(43)  is  general  and  not  specific  to  a  particular  signal  model. 

The  fundamental  difference  between  the  LMS  and  RLS 
algorithms  with  respect  to  their  tracking  performance  can  be 
described  by  evaluating  the  mean  weight  error  vectors  of  the 
two  algorithms.  This  is  obtained  by  examining  the  expected 
value  of  (43).  Using  the  definition  of  the  predictor  error  in 
(11),  the  mean  of  e°k  is  zero.  For  the  LMS  algorithm,  the 
steady-state  solution  to  the  first-order  stochastic  difference 
equation  is  E[ Ak]  «  -(1  /ii)E[xkxl]~lTk.  The  RLS  solution 
was  obtained  in  [8],  where  it  was  shown  that  for  the  slow  chirp 
assumption,  KklKh-i  «  I,  and  the  weight  difference  of  the 
RLS  algorithm  is  thus  E[Ak\  »  -(1  /(3)Tk.  The  optimum 
weight  difference  vector  fk  is  defined  by  (7);  thus,  the  mean 
weight  error  vectors  for  the  LMS  and  RLS  algorithms  are  given 
by 


E[Ak] 


_IVA+i  (,pni>)-1  A  (W°  ©  D)  ,  LMS 
-\v*+1A()V0©L>),  RLS. 

(44) 


The  only  major  difference  is  the  multiplication  by  D-1  in  the 
LMS  mean  lag  weight.  The  effect  of  this  on  the  lag  misadjust- 
ment  is  illustrated  by  decomposing  V  as 


V  =  1  +  pK w  (45) 

where  Rw  =  R  0  (DD11).  Under  the  slow  chirp  condition 
(Mip  <  v),  Rw  is  essentially  the  normalized  correlation  matrix 
of  a  stationary  signal  with  initial  frequency  u). 

To  illustrate  the  effects  of  V~l  on  the  filter  performance,  we 
can  expand  V  using  its  eigeneomponents 

M 

V  =  I  +  p  ^  (46) 

t=l 

where  {cr^U{}i  =  1...M  are  eigenvalues  and  associated 
eigenvectors  of  the  signal  correlation  matrix  R^  .  Since  the 
eigenvectors  {wT}  arc  orthonormal 


LMS  Filter  Response 


Fig.  I .  LMS  filter  transfer  function  over  time  for  a  chirped  ARI  process. 


Prediction  Error  vs.  Adaptation  Constant 


M 


V 


"-■-Et 


p-rfcs,". 

+  per  f 


(47) 


Fig.  2.  Theoretical  and  cxpcrimcnial  oulput  misadjustment  of  the  LMS 
algorithms  for  a  chirped  and  a  stationary  ARI  input,  ploltcd  as  a  ratio  to  the 
Wiener  MSE. 


From  (47),  it  is  easy  to  see  that  the  V~l  essentially  subtracts 
components  of  the  signal  from  the  output.  This  has  the  effect  of 
putting  a  notch  in  the  signal  band  of  the  mean  lag  filter,  as  will 
be  illustrated  later. 

The  differences  exhibited  by  the  mean  lag  in  the  LMS  and 
RLS  algorithm  defined  in  (44)  are  best  illustrated  by  a  plot  of 
the  adaptive  filter  transfer  function.  Fig.  1  and  [8,  Fig.  8],  re¬ 
spectively,  plot  the  components  of  the  filter  transfer  function 
at  three  normalized  adaptation  constants  for  the  L.MS  and  RLS 
algorithms.  The  normalized  adaptation  constants  are  chosen  to 
compare  the  effect  of  the  notch  at  the  optimal  normalized  adap¬ 
tation  constant  */opt  defined  by  (41)  and  normalized  adaptation 
constant  of  2//opt  and  jyopt/2.  Both  the  mean  filter  transfer  func¬ 
tion  and  the  lag  filter  transfer  function  are  shown.  Note  that  the 
difference  in  the  lag  filter  transfer  function  of  the  LMS  and  RLS 


algorithms  results  from  the  effects  of  V~l  shown  in  (47).  This 
produces  a  notching  effect  around  the  signal  spectrum  for  the 
LMS  algorithm  that  is  not  present  in  the  RLS  algorithm. 

Fig.  2  and  [8,  Fig.  9],  respectively,  plot  the  output  misad¬ 
justment  of  the  LMS  and  RLS  algorithms  above  their  respec¬ 
tive  optimum  adaptation  constant  with  parameter  values  p  —  2, 
a  =  0.9,  u  —  0.27T,  ip  =  5  x  10_o7r,  and  M  =  10  for  both 
stationary  and  chirped  input  signals.  The  vertical  lines  indicate 
the  standard  deviation  (±<j)  about  the  simulated  results.  It  can 
be  seen  that  near  the  optimum,  the  experimental  results  agree 
closely  with  the  analytical  results.  When  ;/  >  //0pt,  the  filter 
tracks  the  chirped  signal  with  negligible  lag  error,  and  the  error 
output  is  predominately  noise  misadjustment.  Consequently,  the 
output  misadjustments  for  chirped  and  stationary  ARI  inputs 
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Fig.  3.  Normalized  asymptotic  lag  misadjustment  klms/R« 

versus  SNR  for  a  chirped  ARI  signal  with  correlation  r>  embedded  in  white 
noise.  Results  arc  shown  for  a  between  0.1  and  0.99. 

are  almost  identical.  When  v  <  /'opt,  the  LMS  does  not  track  the 
chirped  signal  well,  and  the  output  error  caused  by  the  lag  mis¬ 
adjustment  dominates  the  performance.  As  a  consequence,  the 
difference  between  output  error  for  the  chirped  and  stationary 
ARI  inputs  increases  exponentially  as  vjv opt  — >  0.  The  ana¬ 
lytical  results  for  the  region  where  the  slow  chirp  assumption 
is  not  valid  are  obtained  numerically.  Note  that  the  simulation 
results  indicate  less  misadjustment  than  the  analysis  predicts. 
These  trends  were  also  observed  for  the  RLS  algorithm  in  [8] 
Additional  comparisons  between  the  analytical  and  simula¬ 
tion  results  are  provided  in  [9].  The  comparative  misadjustment 
of  the  two  algorithms  can  be  plotted  by  substituting  the  optimum 
normalized  adaptation  constants  from  Table  I  into  the  expres¬ 
sion  for  vMopt  and  defining  the  ratio  of  the  optimal  output  mis¬ 
adjustment  for  the  LMS  and  RLS  algorithms 

(Ad mm) LMS  /  ^LMS  \ 

(Admin)RLS  \  *RLS  / 

/(A»lV-(Ar|y  ' 

V  (A\V°)  V{A.W°)  ) 

Fig.  3  and  [8,  Fig.  3],  respectively,  plot  the  asymptotic  normal¬ 
ized  lag  misadjustment  parameter  k  of  the  LMS  and  RLS  algo¬ 
rithms  normalized  by  the  noise  power  ( Pn )  as  the  filter  length 
M  —>  oo.  The  normalized  lag  misadjustment  for  the  two  algo¬ 
rithms  is  equivalent  for  a  between  0. 1  and  0.7,  but  for  a  between 
0.9  and  0.999,  the  differences  in  performance  become  apparent. 
Note  from  [8,  Fig.  3]  that  krls  increases  monotonically  as  a 
function  of  SNR,  whereas  from  Fig.  3,  klms  exhibits  a  non¬ 
monotonic  SNR  response  for  narrowband  signals  with  a  >  0.9. 

Fig.  4  plots  the  ratio  of  the  LMS  misadjustment  versus  the 
RLS  misadjustment  for  an  ARI  process  with  M  =  2  and  10  at 
various  values  of  a  and  p,  leading  to  the  following  observations. 

1)  As  the  signal  bandwidth  increases  (a  — >  0),  the  ratio  of 
the  misadjustment  approaches  1,  i.e.,  in  the  limit  where 


Fig.  4.  Ratio  of  the  LMS  excess  misadjustment  vs.  RLS  excess  misadjustment. 

the  signal  and  noise  have  the  same  bandwidth,  both  algo¬ 
rithms  have  the  same  performance. 

2)  For  small  SNR  (approximately  p  <  10),  the  LMS  al¬ 
gorithm  always  has  a  lower  misadjustment.  As  the  input 
SNR  increases,  however,  the  RLS  algorithm  has  less  error 
for  a  wide  range  of  signal  bandwidths.  In  all  eases,  how¬ 
ever,  there  is  a  range  of  signal  bandwidth  where  LMS 
outperforms  RLS  as  cx  — ►  1.  Note  also  that  the  range  of 
a,  where  LMS  outperforms  RLS,  becomes  smaller  as  the 
SNR  is  increased.  This  property  is  explained  by  the  dif¬ 
ferences  in  the  ratios  of  V  and  V~x  in  (46)  and  (47)  as  a 
function  of  p  and  a  This  behavior  is  confirmed  through 
simulations  in  [9]. 

3)  The  improvements  for  LMS  become  significant  as  a  — ► 
1.  There  is  a  discontinuity  in  the  analysis  for  a  —  1,  and 
the  results  obtained  for  a  deterministic  analysis  under  this 
condition  are  indicated  by  the  bars  at  the  a  =  1  axis  for 
p  =  30,  1 0,  5,  and  l  for  the  eases  of  M  =  2  and  M  =  10. 

IV.  Conclusions 

The  results  show  that  the  comparative  tracking  performance 
of  the  RLS  and  LMS  algorithms  for  time-varying  inputs  such 
as  the  linear  chirp  in  AWGN  is  highly  dependent  on  the  input 
signal  bandwidth  and  signal-to-noise  ratio  p.  Although  both  al¬ 
gorithms  converge  to  the  Wiener  solution  for  stationary  inputs, 
their  performance  in  a  nonstationary  input  environment  can  be 
traced  back  to  their  update  structures.  In  this  paper,  the  update 
structures  of  the  LMS  and  RLS  algorithms  are  examined  It  is 
shown  that  the  update  of  the  LMS  algorithm  inherently  subtracts 
signal  components  from  the  lag  misadjustment.  For  the  chirped 
signal,  it  is  shown  that  this  produces  smaller  tracking  error  for 
small  SNR.  In  the  chirped  ARI  signal  case,  the  LMS  always 
has  smaller  tracking  error  when  p  <  10  dB.  For  p  >  10  dB, 
it  was  shown  that  there  is  a  region  of  signal  bandwidths  where 
RLS  has  superior  performance,  but  even  for  these  high  SNR  in¬ 
puts,  LMS  has  superior  performance  for  very  narrowband  sig¬ 
nals  (a  — ►  1).  The  range  of  bandwidths  for  which  LMS  out- 
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performs  RLS  was  shown  to  be  a  function  of  p.  It  was  further 
shown  that  the  optimal  performance  for  both  the  LMS  and  RLS 
algorithms  was  achieved  by  the  use  of  an  adaptation  constant 
that  provided  a  lag  misadjustment  of  half  the  noise  misadjust- 
ment  for  a  stationary  signal  of  the  same  properties.  For  adapta¬ 
tion  constants  greater  than  this  optimal  value,  the  output  misad¬ 
justment  error  has  a  negligible  component  due  to  lag  error  and 
is  dominated  by  the  noise  misadjustment  term.  Consequently, 
the  performance  of  the  adaptive  filter  for  an  adaptation  constant 
greater  than  the  optimum  is  the  same  for  stationary  and  chirped 
inputs  for  both  the  LMS  and  RLS  algorithms.  For  an  adaptation 
constant  less  than  the  optimum,  the  lag  misadjustment  begins  to 
dominate,  and  there  is  a  significant  difference  in  performance 
for  the  stationary  and  ehirped  inputs. 

Appendix 

In  this  Appendix,  the  quantities  E[x*kx2k  Alk_1(Alk_i)11 
****)>  are  evaluated  for 

the  derivation  from  (33)  to  (35). 

The  following  identity  is  used  in  the  evaluation;  For 
zero-mean  Gaussian  random  vector  x  and  vector  y ,  which  can 
be  either  deterministic  or  random  but,  in  general,  uneorrelated 
with  x 


E  [x*xTyyn x*xT]  =  RxRyRx  4-  RxTr(RxRy) 

where  Rx  =  E[x*x 7],  Ry  =  E[yyu]  for  random  y ,  Ry  = 
yyH  for  deterministic  y. 

A.  Proof 


Applying  the  above  identity,  the  required  quantities  are 


and 


xkxI^k-i  xlxI 

=  3>£Z^  +  $£Tr[^Z*] 


E{xkxlE  [A'  JjqAjUi]"*;*?} 
=HE[^-i]E[Ak^]"**k 
+  E[Ak_l]" 
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